1. Technical Field
The present invention relates to an image pick-up apparatus and an image processing method.
2. Related Art
An image pick-up device such as a CCD image sensor, a CMOS image sensor, etc. is generally provided with color filters above photo acceptance surfaces of photoelectric conversion elements respectively. For example, description will be made in the case where a Bayer arrangement is used as an arrangement of the color filters.
In the case of the Bayer arrangement, G color filters which transmit green (G) light can be classified into two kinds in accordance with the pattern of color filters arranged around each of the G color filters. The first kind is a group of G color filters provided so that R color filters which transmit red (R) light are arranged in the upper and lower of each of the G color filters while B color filters which transmit blue (B) light are arranged in the left and right of each of the G color filters. The second kind is a group of G color filters provided so that B color filters are arranged in the upper and lower of each of the G color filters while R color filters are arranged in the left and right of each of the G color filters.
When color filters transmit light of the same color component but are different in arrangement pattern of adjacent filters in the aforementioned manner, variation in thickness, mask displacement at formation of color filters, etc. cause a difference in light transmittance. Because of the transmittance difference, a sensitivity difference is produced between photoelectric conversion elements located under the two kinds of G color filters respectively. A difference is produced between output signal levels because of the sensitivity difference. Lattice-shaped or horizontal stripe-shaped fixed pattern noise is produced in an image obtained by photographing because of the difference between output signal levels.
FIG. 9 is a graph showing respective output characteristics of two photoelectric conversion elements provided with the aforementioned two kinds of G color filters arranged above the photoelectric conversion elements respectively. As shown in FIG. 9, respective output signal levels of signals GR and GB are r and b with respect to the same incident light intensity, so that a difference (GR-GB level difference) is produced between output signal levels.
To correct the GR-GB level difference, at least one of the output signal level of the signal GR and the output signal level of the signal GB can be multiplied by a correction coefficient so that the characteristics of the signals GR and GB become equal to each other. For example, the following expression (1) having a correction coefficient α is calculated.
By the calculation, as shown in FIG. 10, the characteristic of the signal GB can be changed to characteristic of a signal GB′ substantially equal to the characteristic of the signal GR, so that the difference between the output signal levels can be substantially eliminated. As another correction method, there is a method of calculating an average of output signal levels of signals GR and GB and multiplying the signals GR and GB by correction coefficients respectively to set the average as a goal.The level of signal GB after correction=α×(the level b of signal GB before correction)  (1)in which α=r/b
Because the difference between the output signal levels is caused by optical characteristic, the difference between the output signal levels varies according to the incident angle of light from the photograph lens, the diameter of the iris, the individual difference of the image pick-up device, etc. Accordingly, for example, the maker of the image pick-up apparatus has picked up an image with constant light intensity by using the image pick-up apparatus before shipping of the image pick-up apparatus to calculate the difference between output signal levels based on image pick-up signals obtained thus and calculate a correction coefficient to eliminate the difference so that the image pick-up apparatus can be shipped after the correction coefficient is stored in a memory in the image pick-up apparatus. In this manner, the difference between the output signal levels has been corrected with the correction coefficient determined in accordance with the image pick-up apparatus.
Besides the correction of the difference between the output signal levels, noise reduction processing for reducing random noise components of image pick-up signals is performed by the image pick-up apparatus. Typically, this processing is a process of cutting high-frequency components based on a low pass filter, an averaging process between adjacent pixels, a smoothing process, etc.
In an AD converter mounted in the image pick-up apparatus, analog-to-digital conversion is performed in a state where an OB signal obtained from optical black of an image pick-up device is clamped to a target level so that a predetermined offset is added to the image pick-up signal. For this reason, it is conceived that there are two cases as to whether the noise reduction processing is performed before offset correction for subtracting the offset from the image pick-up signal after AD conversion or whether the noise reduction processing is performed after the offset correction.
It is known that good noise reduction characteristic is obtained when noise reduction processing is performed in a state where the offset is added to the image pick-up signal (see JP-A-2007-110486 and JP-A-2007-312076). It is conceived that execution of noise reduction processing before offset correction is effective for the recent image pick-up apparatus used under a high ISO sensitivity condition.
On the other hand, when the aforementioned GR-GB level difference correction is performed in a state where the offset is added to each of the signals GR and GB, there is a possibility that level difference correction cannot be performed accurately. It is therefore preferable that the GR-GB level difference correction is performed after offset correction. That is, to perform AD conversion, noise reduction, offset correction and GR-GB level difference correction successively on the image pick-up signal is effective.
The noise reduction processing reduces the noise component by averaging the image pick-up signal or the like. Therefore, the aforementioned GR-GB level difference is regarded as signal level variation caused by noise so that the value of the GR-GB level difference varies according to the strength of noise reduction processing. For this reason, there arises a case that accurate correction cannot be performed when offset correction is performed by the method according to the background art after the noise reduction processing.
This case will be described with reference to the drawings. FIG. 11 is a graph showing respective output characteristics of two photoelectric conversion elements provided with the aforementioned two kinds of G color filters arranged above the photoelectric conversion elements respectively. FIG. 11 shows a state where noise components are added to the signals GR and GB whereas FIG. 9 simply expresses the signals GR and GB as lines. FIG. 12 is a graph showing respective output characteristics of two photoelectric conversion elements provided with the aforementioned two kinds of G color filters arranged above the photoelectric conversion elements respectively in the case where noise reduction processing with a predetermined strength is applied to FIG. 11.
As shown in FIG. 12, the GR-GB level difference is changed when the strength of noise reduction processing is changed. For this reason, when a fixed correction coefficient α is determined so that the GR-GB level difference shown in FIG. 11 can be corrected accurately, characteristic of the corrected signal GB shown in FIG. 12 is changed to characteristic of a signal GB′ shown in FIG. 13 to thereby cause over-correction (mistaken correction) that the level of the signal GB′ exceeds the level of the signal GR. Because the strength of noise reduction processing is changed independently of optical characteristic such as iris, zooming, etc. but dependently on analog gain such as ISO sensitivity, such mistaken correction is apt to be found under a high ISO sensitivity photographing condition that noise reduction is emphasized particularly strongly.
JP-A-11-191889 and JP-A-2000-341700 have disclosed methods of performing correction with a fixed correction coefficient to equalize signal levels of pixels of the same color. It is however impossible for these methods to fundamentally address the case of occurrence of mistaken correction because correction is always performed with a fixed correction coefficient.